LogStash: A great way to spend your Time getting to know ELK and Logging in general
We investigated (and use) Logstash as apart of our ELK (Elasticsearch, Logstash and Kibana) stack. We input data from websites that post …
Overview
Recent Reviews
Pricing
Entry-level set up fee?
- No setup fee
Offerings
- Free Trial
- Free/Freemium Version
- Premium Consulting/Integration Services
Would you like us to let the vendor know that you want pricing?
6 people also want pricing
Alternatives Pricing
What is Datadog?
Datadog is a monitoring service for IT, Dev and Ops teams who write and run applications at scale, and want to turn the massive amounts of data produced by their apps, tools and services into actionable insight.
N/A
Unavailable
What is LogicMonitor?
LogicMonitor provides an agentless SaaS-based monitoring platform. LogicMonitor provides prebuilt integrations and an open API, and is designed to provide monitoring across networks, servers, applications, websites, and containers, including insights and reporting capabilities.
Product Demos
logstash demo (hundreds of servers, 7-node cluster, websockets, ruby hacks)
YouTube
Arista/Kafka/Elastic integration demo
YouTube
logstash demo piping log files to websockets.
YouTube
Kibana, Logstash, Elastic - a quick demo of the visualisation tools
YouTube
Real TIme Python Log Ingestion with Logstash & elk and Visualize Logs on Kibana | Demo & Code
YouTube
Install Elasticsearch, Kibana, Logstash, and Filebeat using Docker Compos
YouTube
Product Details
- About
- Tech Details
What is Logstash?
Logstash Technical Details
| Operating Systems | Unspecified |
|---|---|
| Mobile Application | No |
Comparisons
Compare with
Reviews and Ratings
(21)Reviews
(1-2 of 2)Companies can't remove reviews or game the system. Here's why
My primary use case for Logstash is ingesting log files into a local Elasticsearch&Kibana Docker container so that I can easily search though the logs better. My favorite feature is the grok parser as it is easy to decompose complex regular expressions into simplified patterns. Logstash has a plethora of available plugins, but the out of the box connections have addressed all my needs thus far.
- Plugin ecosystem allows modular extensions.
- Tight integration into the Elastic.com products of Beats and Elasticsearch, so minimal setup is required when using those tools.
- Filter plugins are powerful for extracting and enriching input data.
- Since it's a Java product, JVM tuning must be done for handling high-load.
- The persistent queue feature is nice, but I feel like most companies would want to use Kafka as a general storage location for persistent messages for all consumers to use. Using some pipeline of "Kafka input -> filter plugins -> Kafka output" seems like a good solution for data enrichment without needing to maintain a custom Kafka consumer to accomplish a similar feature.
- I would like to see more documentation around creating a distributed Logstash cluster because I imagine for high ingestion use cases, that would be necessary.
- Logstash has allowed me to ingest log files of various patterns into Elasticsearch for analysis using its flexible Grok parser.
- I've been able to perform web analytics over datasets using Logstash's GeoIP and reverse DNS lookups.
- By providing a simple mechanism for adding plugins, Logstash has allowed me to install extensions on top of those already pre-installed.
Logstash can be compared to other ETL frameworks or tools, but it is also complementary to several, for example, Kafka. I would not only suggest using Logstash when the rest of the ELK stack is available, but also for a self-hosted event collection pipeline for various searching systems such as Solr or Graylog, or even monitoring solutions built on top of Graphite or OpenTSDB.
March 15, 2018
Logstash is the best for ELK stacks!
We were introduced to Logstash via the ELK stack. Our application generates many data points, and one of many patterns we had seen was to store it in elastic search. Logstash was the router which actually sent data to ES, and received data from our applications. The power of Logstash was quickly realized when we increased the number of source applications generating different kinds of data, but the ingestion point remained same. Since then Logstash has been one of my team's favorite tools and is one of the few things which you can set once and forget. We are happy users of Logstash and expect to use it more in the future.
- Logstash design is definitely perfect for the use case of ELK. Logstash has "drivers" using which it can inject from virtually any source. This takes the headache from source to implement those "drivers" to store data to ES.
- Logstash is fast, very fast. As per my observance, you don't need more than 1 or 2 servers for even big size projects.
- Data in different shape, size, and formats? No worries, Logstash can handle it. It lets you write simple rules to programmatically take decisions real-time on data.
- You can change your data on the fly! This is the CORE power of Logstash. The concept is similar to Kafka streams, the difference being the source and destination are application and ES respectively.
- Logstash is all command line, and it can become overwhelming for new developers. If it has any sort of UI, then I don't know about it.
- Documentation could have been better. But this is a work in progress, and with time I am sure community will help with documentation.
- Community support! Being a relatively new tool, the adoption is still mature, and finding answers can be challenging sometimes.
- Positive: Learning curve was relatively easy for our team. We were up and running within a sprint.
- Positive: Managing Logstash has generally been easy. We configure it, and usually, don't have to worry about misbehavior.
- Negative: Updating/Rehydrating Logstash servers have been little challenging. We sometimes even loose data while Logstash is down. It requires more in-depth research and experiments to figure the fine-grained details.
- Negative: This is now one more application/skill/server to manage. Like any other servers, it requires proper grooming or else you will get in trouble. This is also a single point of failure which can have the ability to make other servers useless if it is not running.
Logstash is similar to any service which can be the single point to collect and transform data. Kafka is a very good candidate, but it fails for applications not using Kafka. Kafka streams do pretty much the same thing. On one hand, I personally trust Kafka more, but then Kafka can only handle Kafka messages, whereas Logstash, can inject from any source e.g. databases. So any application can send data to Logstash. Another benefit of Logstash over Kafka is the ability to add plugins. If there is some source which is not supported by core Logstash, you can add other plugins.